Method and apparatus for using gestures to control a laser tracker

ABSTRACT

A method and system are provided for controlling a laser tracker remotely from the laser tracker through gestures performed by a user. The method includes providing a rule of correspondence between each of a plurality of commands and each of a plurality of user gestures. A gesture is performed by the user with the user&#39;s body that corresponds to one of the plurality of user gestures. The gesture performed by the user is detected. The gesture recognition engine determines a first command from one of the plurality of commands that correspond with the detected gesture. Then the first command is executed with the laser tracker.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S.application Ser. No. 14/803,575 entitled “Method and Apparatus for UsingGestures to Control a Laser Tracker,” which is a divisional applicationof U.S. application Ser. No. 14/264,420 entitled “Method and Apparatusfor Using Gestures to Control a Laser Tracker.” U.S. application Ser.No. 14/264,420 is a Nonprovisional Application of U.S. ProvisionalApplication Ser. No. 61/818,208 filed on May 1, 2013 entitled “Methodand Apparatus for Using Gestures to Control a Laser Tracker,” thecontents of both of which are incorporated by reference herein.

BACKGROUND OF THE INVENTION

The present disclosure relates to a coordinate measuring device. One setof coordinate measurement devices belongs to a class of instruments thatmeasure the three-dimensional (3D) coordinates of a point by sending alaser beam to the point, where it is intercepted by a retroreflectortarget. The instrument finds the coordinates of the point by measuringthe distance and the two angles to the target. The distance is measuredwith a distance-measuring device such as an absolute distance meter(ADM) or an interferometer. The angles are measured with anangle-measuring device such as an angular encoder. A gimbaledbeam-steering mechanism within the instrument directs the laser beam tothe point of interest. An example of such a device is a laser tracker.Exemplary laser tracker systems are described by U.S. Pat. No. 4,790,651to Brown et al., the contents of which is incorporated by referenceherein, and U.S. Pat. No. 4,714,339 to Lau et al.

A coordinate-measuring device closely related to the laser tracker isthe total station. The total station, which is most often used insurveying applications, may be used to measure the coordinates ofdiffusely scattering or retroreflective targets. Hereinafter, the termlaser tracker is used in a broad sense to include total stations.

Ordinarily the laser tracker sends a laser beam to a retroreflectortarget. A common type of retroreflector target is the sphericallymounted retroreflector (SMR), which comprises a cube-cornerretroreflector embedded within a metal sphere. The cube-cornerretroreflector comprises three mutually perpendicular mirrors. The apexof the cube corner, which is the common point of intersection of thethree mirrors, is located at the center of the sphere. It is commonpractice to place the spherical surface of the SMR in contact with anobject under test and then move the SMR over the surface being measured.Because of this placement of the cube corner within the sphere, theperpendicular distance from the apex of the cube corner to the surfaceof the object under test remains constant despite rotation of the SMR.Consequently, the 3D coordinates of a surface can be found by having atracker follow the 3D coordinates of an SMR moved over the surface. Itis possible to place a glass window on the top of the SMR to preventdust or dirt from contaminating the glass surfaces.

A gimbal mechanism within the laser tracker may be used to direct alaser beam from the tracker to the SMR. Part of the light retroreflectedby the SMR enters the laser tracker and passes onto a position detector.The position of the light that hits the position detector is used by atracker control system to adjust the rotation angles of the mechanicalazimuth and zenith axes of the laser tracker to keep the laser beamcentered on the SMR. In this way, the tracker is able to follow (track)the SMR.

Angular encoders attached to the mechanical azimuth and zenith axes ofthe tracker may measure the azimuth and zenith angles of the laser beam(with respect to the tracker frame of reference). The one distancemeasurement and two angle measurements performed by the laser trackerare sufficient to completely specify the three-dimensional location ofthe SMR.

As mentioned previously, two types of distance meters may be found inlaser trackers: interferometers and absolute distance meters (ADMs). Inthe laser tracker, an interferometer (if present) may determine thedistance from a starting point to a finishing point by counting thenumber of increments of known length (usually the half-wavelength of thelaser light) that pass as a retroreflector target is moved between thetwo points. If the beam is broken during the measurement, the number ofcounts cannot be accurately known, causing the distance information tobe lost. By comparison, the ADM in a laser tracker determines theabsolute distance to a retroreflector target without regard to beambreaks, which also allows switching between targets. Because of this,the ADM is said to be capable of “point-and-shoot” measurement.Initially, absolute distance meters were only able to measure stationarytargets and for this reason were always used together with aninterferometer. However, some modern absolute distance meters can makerapid measurements, thereby eliminating the need for an interferometer.Such an ADM is described in U.S. Pat. No. 7,352,446 to Bridges et al.,the contents of which is incorporated by reference herein.

In its tracking mode, the laser tracker will automatically followmovements of the SMR when the SMR is in the capture range of thetracker. If the laser beam is broken, tracking will stop. The beam maybe broken by any of several means: (1) an obstruction between theinstrument and SMR; (2) rapid movements of the SMR that are too fast forthe instrument to follow; or (3) the direction of the SMR being turnedbeyond the acceptance angle of the SMR. By default, following the beambreak, the beam remains fixed at the point of the beam break or at thelast commanded position. The operator may visually search for thetracking beam and place the SMR in the beam in order to lock theinstrument onto the SMR and continue tracking.

Some laser trackers include one or more cameras. A camera axis may becoaxial with the measurement beam or offset from the measurement beam bya fixed distance or angle. A camera may be used to provide a wide fieldof view to locate retroreflectors. A modulated light source placed nearthe camera optical axis may illuminate retroreflectors, thereby makingthem easier to identify. In this case, the retroreflectors flash inphase with the illumination, whereas background objects do not. Oneapplication for such a camera is to detect multiple retroreflectors inthe field of view and measure each in an automated sequence. Exemplarysystems are described in U.S. Pat. No. 6,166,809 to Pettersen et al.,and U.S. Pat. No. 7,800,758 to Bridges et al., the contents of which areincorporated by reference herein.

Some laser trackers have the ability to measure with six degrees offreedom (DOF), which may include three coordinates, such as x, y, and z,and three rotations, such as pitch, roll, and yaw. Several systems basedon laser trackers are available or have been proposed for measuring sixdegrees of freedom. Exemplary systems are described in U.S. PublishedPatent Application No. 2010/0128259 to Bridges, the contents of which isincorporated by reference herein; U.S. Pat. No. 7,800,758 to Bridges etal., U.S. Pat. No. 5,973,788 to Pettersen et al.; and U.S. Pat. No.7,230,689 to Lau.

Two common modes of operation of the laser tracker are tracking mode andprofiling mode. In tracking mode, the laser beam from the trackerfollows the retroreflector as the operator moves it around. In profilingmode, the laser beam from the tracker goes in the direction given by theoperator, either through computer commands or manual action.

Besides these modes of operation that control the basic tracking andpointing behavior of the tracker, there are also other option modes thatenable the tracker to respond in a manner selected by the operator aheadof time. The desired option mode is typically selected in software thatcontrols the laser tracker. Such software may reside in an externalcomputer attached to the tracker (possibly through a network cable) orwithin the tracker itself. In the latter case, the software may beaccessed through console functionality built into the tracker.

An example of an option mode is the Auto Reset mode in which the laserbeam is driven to a preset reference point whenever the laser beam isbroken. One popular reference point for the Auto Reset option mode isthe tracker Home Position, which is the position of a magnetic nestmounted on the tracker body. The alternative to Auto Reset is the NoReset option mode. In this case, the laser beam continues pointing inthe original direction whenever the laser beam is broken. A descriptionof the tracker home position is given in U.S. Pat. No. 7,327,446 toCramer et al., the contents of which is incorporated by referenceherein.

Another example of an option mode is sometimes referred to as“PowerLock”. In the PowerLock option mode, the location of theretroreflector is found by a tracker camera whenever the tracker laserbeam is broken. The camera immediately sends the angular coordinates ofthe retroreflector to the tracker control system, thereby causing thetracker to point the laser beam back at the retroreflector. Methodsinvolving automatic acquisition of a retroreflector are given ininternational application WO 2007/079601 to Dold et al. and U.S. Pat.No. 7,055,253 to Kaneko.

Some option modes are slightly more complex in their operation. Anexample is the Stability Criterion mode, which may be invoked wheneveran SMR is stationary for a given period of time. The operator may trackan SMR to a magnetic nest and set it down. If a stability criterion isactive, the software will begin to look at the stability of thethree-dimensional coordinate readings of the tracker. For instance, theuser may decide to judge the SMR to be stable if the peak-to-peakdeviation in the distance reading of the SMR is less than twomicrometers over a one second interval. After the stability criterion issatisfied, the tracker measures the 3D coordinates and the softwarerecords the data.

More complex modes of operation are possible through computer programs.For example, software is available to measure part surfaces and fitthese to geometrical shapes. Software will instruct the operator to movethe SMR over the surface and then, when finished collecting data points,to raise the SMR off the surface of the object to end the measurement.Moving the SMR off the surface not only indicates that the measurementis completed; it also indicates the position of the SMR in relation tothe object surface. This position information is needed by theapplication software to account for the offset caused by the SMR radius.

A second example of computer control is a tracker survey. In the survey,the tracker is driven sequentially to each of several target locationsaccording to a prearranged schedule. The operator may teach thesepositions prior to the survey by carrying the SMR to each of the desiredpositions.

A third example of software control is tracker directed measurement. Thesoftware directs the operator to move the SMR to a desired location. Itdoes this using a graphic display to show the direction and distance tothe desired location. When the operator is at the desired position, thecolor on the computer monitor might, for example, turn from red togreen.

The element common to the tracker actions described above is that theoperator is limited in his ability to control the behavior of thetracker. On the one hand, option modes selected in the software mayenable the operator to preset certain behaviors of the tracker. However,once the option modes have been selected by the user, the behavior ofthe tracker is established and cannot be changed unless the operatorreturns to the computer console. On the other hand, the computer programmay direct the operator to carry out complicated operations that thesoftware analyzes in a sophisticated way. In either case, the operatoris limited in his ability to control the tracker and the data collectedby the tracker.

A laser tracker operator performs two different functions. The operatorpositions an SMR during a measurement, and sends commands through thecontrol computer to the tracker. However, it is not easy for a singleoperator to perform both of these measurement functions because thecomputer is usually remotely located from the measurement location.Various methods have been tried to get around this limitation, but noneis completely satisfactory.

One method sometimes used is for a single operator to set theretroreflector in place and walk back to the instrument control keyboardto execute a measurement instruction. However, this is an inefficientuse of operator and instrument time. In cases where the operator musthold the retroreflector for the measurement, single operator control isonly possible when the operator is very close to the keyboard.

A second method is to add a second operator. One operator stands by thecomputer and a second operator moves the SMR. This is labor intensiveand an expensive method. Further, verbal communication over largedistances can be a problem.

A third method is to equip a laser tracker with a remote control.However, remote controls have several limitations. Many facilities donot allow the use of remote controls for safety or security reasons.Even if remote controls are allowed, interference among wirelesschannels may be an issue. Some remote control signals do not reach thefull range of the laser tracker. In some situations, such as workingfrom a ladder, the second hand may not be free to operate the remotecontrol. Before a remote control can be used, it is usually necessary toset up the computer and remote control to work together, and then only asmall subset of tracker commands can ordinarily be accessed at any giventime. An example of a system based on this idea is given in U.S. Pat.No. 7,233,316 to Smith et al.

A fourth method is to interface a cell phone to a laser tracker.Commands are entered remotely by calling the instrument from the cellphone and entering numbers from the cell phone keypad or by means ofvoice recognition. This method also has many shortcomings. Somefacilities do not allow cell phones to be used, and cell phones may notbe available in rural areas. Service requires a monthly service providerfee. A cell phone interface requires additional hardware interfacing tothe computer or laser tracker. Cell phone technology changes fast andmay require upgrades. As in the case of remote controls, the computerand remote control must be set up to work together, and only a smallsubset of tracker commands can ordinarily be accessed at a given time.

A fifth method is to equip a laser tracker with internet or wirelessnetwork capabilities and use a wireless portable computer or personaldigital assistant (PDA) to communicate commands to the laser tracker.However, this method has limitations similar to a cell phone. Thismethod is often used with total stations. Examples of systems that usethis method include U.S. Published Patent Application No. 2009/017618 toKumagai et al., U.S. Pat. No. 6,034,722 to Viney et al., U.S. Pat. No.7,423,742 to Gatsios et al., U.S. Pat. No. 7,307,710 to Gatsios et al.,U.S. Pat. No. 7,552,539 to Piekutowski, and U.S. Pat. No. 6,133,998 toMonz et al. This method has also been used to control appliances by amethod described in U.S. Pat. No. 7,541,965 to Ouchi et al.

A sixth method is to use a pointer to indicate a particular location inwhich a measurement is to be made. An example of this method is given inU.S. Pat. No. 7,022,971 to Ura et al. It might be possible to adapt thismethod to give commands to a laser tracker, but it is not usually veryeasy to find a suitable surface upon which to project the pointer beampattern.

A seventh method is to devise a complex target structure containing atleast a retroreflector, transmitter, and receiver. Such systems may beused with total stations to transmit precise target information to theoperator and also to transmit global positioning system (GPS)information to the total station. An example of such a system is givenin U.S. Published Patent Application No. 2008/0229592 to Hinderling etal. In this case, no method is provided to enable the operator to sendcommands to the measurement device (total station).

An eighth method is to devise a complex target structure containing atleast a retroreflector, transmitter and receiver, where the transmitterhas the ability to send modulated light signals to a total station. Akeypad can be used to send commands to the total station by means of themodulated light. These commands are decoded by the total station.Examples of such systems are given in U.S. Pat. No. 6,023,326 toKatayama et al., U.S. Pat. No. 6,462,810 to Muraoka et al., U.S. Pat.No. 6,295,174 to Ishinabe et al., and U.S. Pat. No. 6,587,244 toIshinabe et al. This method is particularly appropriate for surveyingapplications in which the complex target and keypad are mounted on alarge staff. Such a method is not suitable for use with a laser tracker,where it is advantageous to use a small target not tethered to a largecontrol pad. Also it is desirable to have the ability to send commandseven when the tracker is not locked onto a retroreflector target.

A ninth method is to include both a wireless transmitter and a modulatedlight source on the target to send information to a total station. Thewireless transmitter primarily sends information on the angular pose ofthe target so that the total station can turn in the proper direction tosend its laser beam to the target retroreflector. The modulated lightsource is placed near the retroreflector so that it will be picked up bythe detector in the total station. In this way, the operator can beassured that the total station is pointed in the right direction,thereby avoiding false reflections that do not come from the targetretroreflector. An exemplary system based on this approach is given inU.S. Pat. No. 5,313,409 to Wiklund et al. However, this method does notoffer the ability to send general purpose commands to a laser tracker.

A tenth method is to include a combination of wireless transmitter,compass assembly in both target and total station, and guide lighttransmitter. The compass assembly in the target and total station areused to enable alignment of the azimuth angle of the total station tothe target. The guide light transmitter is a horizontal fan of lightthat the target can pan in the vertical direction until a signal isreceived on the detector within the total station. Once the guide lighthas been centered on the detector, the total station adjusts itsorientation slightly to maximize the retroreflected signal. The wirelesstransmitter communicates information entered by the operator on a keypadlocated at the target. An exemplary system based on this method is givenin U.S. Pat. No. 7,304,729 to Wasutomi et al. However, this method doesnot offer the ability to send general purpose commands to a lasertracker.

An eleventh method is to modify the retroreflector to enable temporalmodulation to be imposed on the retroreflected light, therebytransmitting data. The inventive retroreflector comprises a cube cornerhaving a truncated apex, an optical switch attached to the front face ofthe cube corner, and electronics to transmit or receive data. Anexemplary system of this type is given in U.S. Pat. No. 5,121,242 toKennedy. This type of retroreflector is complex and expensive. Itdegrades the quality of the retroreflected light because of the switch(which might be a ferro-electric light crystal material) and because ofthe truncated apex. Also, the light returned to a laser tracker isalready modulated for use in measuring the ADM beam, and switching thelight on and off would be a problem, not only for the ADM, but also forthe tracker interferometer and position detector.

A twelfth method is to use a measuring device that containsbidirectional transmitter for communicating with a target and an activeretroreflector to assist in identifying the retroreflector. Thebidirectional transmitter may be wireless or optical and is part of acomplex target staff that includes the retroreflector, transmitter, andcontrol unit. An exemplary system of this type is described in U.S. Pat.No. 5,828,057 to Hertzman et al. Such a method is not suitable for usewith a laser tracker, where it is advantageous to use a small target nottethered to a large control pad. Also the method of identifying theretroreflector target of interest is complicated and expensive.

Accordingly, while existing laser trackers are suitable for theirintended purposes the need for improvement remains. In particular thereis a need for a simple method for an operator to communicate commands toa laser tracker from a distance.

SUMMARY OF THE INVENTION

In accordance with an embodiment of the invention, a method forcommunicating, from a user to a laser tracker, a way to controloperation of the laser tracker is provided. The method includesproviding the laser tracker having a structure rotatable about a firstaxis and a second axis, a first light source that launches a first lightbeam from the structure, a distance meter, a first angular encoder thatmeasures a first angle of rotation about the first axis, a secondangular encoder that measures a second angle of rotation about thesecond axis, and a processor. A second light source and a camera areprovided, the second light source configured to project a second lightbeam, the camera configured to acquire an image of the second light beamreflected off of a user. A rule of correspondence is provided betweeneach of a plurality of commands and each of a plurality of gestures,each gesture from among the plurality of gestures corresponding to abody position of the user. The second light beam is provided and a firstimage of the user is acquired with the camera. A skeletal model of theuser is generated from the first image. A first gesture is performed bythe user from among the plurality of gestures, the first gesturecorresponding to a first body position. The first gesture is identifiedfrom the skeletal model. The first command is determined based at leastin part on the identifying of the first gesture according to the rule ofcorrespondence. The first command is executed with the laser tracker.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter, which is regarded as the invention, is particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The foregoing and other features, and advantages ofthe invention are apparent from the following detailed description takenin conjunction with the accompanying drawings in which:

FIG. 1, including FIGS. 1A-1C show perspective views of exemplary lasertrackers;

FIG. 2 shows computing and power supply elements attached to the lasertracker of FIG. 1;

FIG. 3 is a block diagram an electronics processing system associatedwith the laser tracker of FIG. 1;

FIG. 4 is a perspective view of the laser tracker of FIG. 1 with theoperator using gestures to control operation;

FIG. 5 is an illustration of the operator making a gesture to record ameasurement;

FIG. 6 is a flow diagram illustrating an operation of the laser trackerwith a gesture;

FIGS. 7-9 show a selection of laser tracker commands and correspondinggestures that may be used by the operator to convey these commands tothe laser tracker;

FIG. 10 is a flow diagram illustrating the use of gestures to set atracker reference point;

FIG. 11 is a flow diagram illustrating the use of gestures to measure acircle;

FIG. 12 is a flow diagram illustrating the use of gestures to acquire aretroreflector with a laser beam from a laser tracker;

FIG. 13 is a perspective view of a laser tracker in accordance withanother embodiment of the invention;

FIG. 14 is a perspective view of the operator's arm with a wearablegesture device for use with the laser tracker of FIG. 13;

FIG. 15 is an illustration of the tracking of a user and the use ofgestures in conveying commands to a laser tracker; and

FIG. 16 is a flow diagram illustrating the tracking of the user and theuse of gestures to convey commands to a laser tracker.

The detailed description explains embodiments of the invention, togetherwith advantages and features, by way of example with reference to thedrawings.

DETAILED DESCRIPTION OF THE EMBODIMENTS

An exemplary laser tracker 10 is illustrated in FIG. 1. An exemplarygimbaled beam-steering mechanism 12 of laser tracker 10 comprises zenithcarriage 14 mounted on azimuth base 16 and rotated about azimuth axis20. Payload 15 is mounted on zenith carriage 14 and rotated about zenithaxis 18. Zenith mechanical rotation axis 18 and azimuth mechanicalrotation axis 20 intersect orthogonally, internally to tracker 10, atgimbal point 22, which is typically the origin for distancemeasurements. Laser beam 46 virtually passes through gimbal point 22 andis pointed orthogonal to zenith axis 18. In other words, laser beam 46is in the plane normal to zenith axis 18. Laser beam 46 is pointed inthe desired direction by motors within the tracker (not shown) thatrotate payload 15 about zenith axis 18 and azimuth axis 20. Zenith andazimuth angular encoders, internal to the tracker (not shown), arecoupled to zenith mechanical axis 18 and azimuth mechanical axis 20 andindicate, to high accuracy, the angles of rotation. Laser beam 46travels to external retroreflector 26 such as a spherically mountedretroreflector (SMR) 26. By measuring the radial distance between gimbalpoint 22 and retroreflector 26 and the rotation angles about the zenithand azimuth axes 18, 20, the position of retroreflector 26 is foundwithin the spherical coordinate system of the tracker 10.

Laser beam 46 may comprise one or more laser wavelengths. For clarityand simplicity, a steering mechanism of the sort shown in FIG. 1 isassumed in the following discussion and the claimed invention should notbe so limited. In other embodiments different types of steeringmechanisms are possible. For example, it would be possible to reflect alaser beam off a mirror rotated about the azimuth and zenith axes. Anexample of the use of a mirror in this way is given in U.S. Pat. No.4,714,339 to Lau et al. The techniques described here are applicable,regardless of the type of steering mechanism.

In exemplary laser tracker 10, cameras 52 and light sources 54 arelocated on payload 15. Light sources 54 illuminate one or moreretroreflector targets 26. Light sources 54 may be LEDs electricallydriven to repetitively emit pulsed light. Each camera 52 comprises aphotosensitive array and a lens placed in front of the photosensitivearray. The photosensitive array may be a CMOS or CCD array. The lens mayhave a relatively wide field of view, say thirty or forty degrees. Thepurpose of the lens is to form an image on the photosensitive array ofobjects within the field of view of the lens. Each light source 54 isplaced near camera 52 so that light from light source 54 is reflectedoff each retroreflector target 26 onto camera 52. In this way,retroreflector images are readily distinguished from the background onthe photosensitive array as their image spots are brighter thanbackground objects and are pulsed. There may be two cameras 52 and twolight sources 54 placed about the line of laser beam 46. By using twocameras in this way, the principle of triangulation can be used to findthe three-dimensional coordinates of any SMR within the field of view ofthe camera. In addition, the three-dimensional coordinates of the SMRcan be monitored as the SMR is moved from point to point. A use of twocameras for this purpose is described in commonly owned U.S. PublishedPatent Application No. 2010/0128259 to Bridges which is incorporated byreference herein.

Other arrangements of one or more cameras and light sources arepossible. For example, a light source and camera can be coaxial ornearly coaxial with the laser beams emitted by the tracker. In thiscase, it may be necessary to use optical filtering or similar methods toavoid saturating the photosensitive array of the camera with the laserbeam from the tracker.

Another possible arrangement is to use a single camera located on thepayload or base of the tracker. A single camera, if located off theoptical axis of the laser tracker, provides information about the twoangles that define the direction to the retroreflector but not thedistance to the retroreflector. In many cases, this information may besufficient. If the 3D coordinates of the retroreflector are needed whenusing a single camera, one possibility is to rotate the tracker in theazimuth direction by 180 degrees and then to flip the zenith axis topoint back at the retroreflector. In this way, the target can be viewedfrom two different directions and the 3D position of the retroreflectorcan be found using triangulation.

A more general approach to finding the distance to a retroreflector witha single camera is to rotate the laser tracker about either the azimuthaxis or the zenith axis and observe the retroreflector with a cameralocated on the tracker for each of the two angles of rotation. Theretroreflector may be illuminated, for example, by an LED located closeto the camera. Another possibility is to switch between measuring andimaging of the target. An example of such a method is described in U.S.Pat. No. 7,800,758 to Bridges et al. Other camera arrangements arepossible and can be used with the methods described herein.

In the exemplary embodiment, the laser tracker 10 further includes agesture capture device 55. The gesture capture device 55 includes aprojector 57 and at least one camera 59. In one embodiment, the gesturecapture device 55 is disposed within the payload 15 as shown in FIG. 1A.It should be appreciated that the projector 57 and camera 59 may bedisposed adjacent the output aperture 61, such as adjacent the lightsources 54 for example. One advantage of placing the projector 57 andcamera 59 on the payload 15 is improved spatial resolution since it ispossible to use a camera having a narrow field of view (FOV) incombination with a payload 15 that can be rotated over a wide range ofaccurately measured angles.

In another embodiment shown in FIG. 1B, the gesture capture device 55 isdisposed within a housing 63. In this case, the gesture capture device55 rotates about the azimuth axis 20 but remains at a fixed locationrelative to the zenith axis 18. In an embodiment, the gesture capturedevice 55 includes a second camera 65. In an embodiment, the projector57 may be an infrared projector, the first camera 59 an infrared camera,and the second camera 65 an RGB camera (capable of capturing red, green,and blue colors).

Still another embodiment is shown in FIG. 1C where the gesture capturedevice 55 is coupled to the base 16. In this embodiment, the gesturecapture device 55 is fixed relative to both the azimuth axis 20 and thezenith axis 18.

Although FIGS. 1A, 1B, and 1C show the gesture capture device affixed tothe tracker, it should be understood that a separate gesture capturedevice may be detached from the tracker and used to capture gestures.Such a separate gesture device may provide received information tointernal processors with the laser tracker 10 or to an externalcomputer, such as computer 80 shown in FIG. 2.

The gesture capture device 55 measures coordinates in three dimensions.In the exemplary embodiment, the gesture capture device 55 is astructured light scanner. A structured light scanner is configured toemit a structured light over an area, such as the area occupied by theoperator. As used herein, the term “structured light” refers to atwo-dimensional pattern of light projected onto an area of an objectthat conveys information which may be used to determine coordinates ofpoints in the field of view, such as the operator for example. Thestructured light pattern is not limited to a easily defined pattern butmay include more complex patterns such as speckle patterns, for example.As will be discussed in more detail herein, the gesture capture device55 measures one or more three-dimensional coordinates to determine ifthe operator is performing a gesture corresponding to a command. Thegesture may be in the form of a body position, such as a movement of ahand for example, or a body position, such as the arrangement of a handfor example.

In general, there are two types of structured light patterns, a codedlight pattern and an uncoded light pattern. As used herein a coded lightpattern is one in which the three dimensional coordinates of anilluminated surface of the object are found by acquiring a single image.With a coded light pattern, it is possible to obtain and register pointcloud data while the projecting device is moving relative to the object.One type of coded light pattern contains a set of elements (e.g.geometric shapes) arranged in lines where at least three of the elementsare non-collinear. Such pattern elements are recognizable because oftheir arrangement. Another type of coded light pattern is nearly randomin its appearance and may be evaluated through the use of correlationmethods, for example.

In contrast, an uncoded structured light pattern as used herein is apattern that does not allow measurement through a single pattern. Aseries of uncoded light patterns may be projected and imagedsequentially. For this case, it is usually necessary during themeasurement to hold the projector substantially fixed relative to theobject.

It should be appreciated that the scanner 20 may use either coded oruncoded structured light patterns. The structured light pattern mayinclude the patterns disclosed in the journal article “DLP-BasedStructured Light 3D Imaging Technologies and Applications” by Jason Gengpublished in the Proceedings of SPIE, Vol. 7932, which is incorporatedherein by reference.

Referring now to FIG. 2, an embodiment is shown of a laser tracker 10having an auxiliary unit 70. The auxiliary unit 70 supplies electricalpower to the laser tracker 10 and in some cases also provides computingand clocking capability. In one embodiment, the separate auxiliary unit70 is eliminated by moving the functionality of auxiliary unit 70 intothe tracker base 16. In most cases, auxiliary unit 70 is attached togeneral purpose computer 80. Application software loaded onto generalpurpose computer 80 may provide application capabilities such as reverseengineering. It is also possible to eliminate general purpose computer80 by building its computing capability directly into laser tracker 10.In this case, a user interface, possibly providing keyboard and mousefunctionality is built into laser tracker 10. The connection betweenauxiliary unit 70 and computer 80 may be wireless, such as through Wi-Fior Bluetooth communications, for example, or be wired through a cable ofelectrical wires, such as a serial, coaxial or Ethernet cable forexample. Computer 80 may be connected to a network, and auxiliary unit70 may also be connected to a network. In one embodiment, theapplication software is operated in a distributed computing environment.It should be appreciated that the computer 80 may be directly coupled tothe auxiliary unit 70, or may be remote from the laser tracker 10 andconnected via a local or wide area network. Plural instruments, such asmultiple measurement instruments or actuators for example, may beconnected together, either through computer 80 or auxiliary unit 70.

The laser tracker 10 may be rotated on its side, rotated upside down, orplaced in an arbitrary orientation. In these situations, the termsazimuth axis and zenith axis have the same direction relative to thelaser tracker as the directions shown in FIG. 1 regardless of theorientation of the laser tracker 10.

In another embodiment, the payload 15 is replaced by a mirror thatrotates about the azimuth axis 20 and the zenith axis 18. A laser beamis directed upward and strikes the mirror, from which it launches towarda retroreflector 26. In still another embodiment, the payload 15 may bereplaced by a two or more galvanometer mirrors that are rotatedindependently of each other to direct the laser beam to the desiredlocation.

The methods for operating the laser tracker 10 discussed herein may beimplemented by means of processing system 800 shown in FIG. 3.Processing system 800 comprises tracker processing unit 810 andoptionally computer 80. Processing unit 810 includes at least oneprocessor, which may be a microprocessor, digital signal processor(DSP), field programmable gate array (FPGA), or similar device.Processing capability is provided to process information and issuecommands to internal tracker processors. Such processors may includeposition detector processor 812, azimuth encoder processor 814, zenithencoder processor 816, indicator lights processor 818, ADM processor820, interferometer (IFM) processor 822, and camera processor 824. Theprocessing unit 810 may further include a gesture recognition engine 826to assist in evaluating or parsing of gestures patterns. Auxiliary unitprocessor 870 optionally provides timing and microprocessor support forother processors within tracker processor unit 810. It may communicatewith other processors by means of device bus 830, which may transferinformation throughout the tracker by means of data packets, as is wellknown in the art. Computing capability may be distributed throughouttracker processing unit 810, with DSPs and FPGAs performing intermediatecalculations on data collected by tracker sensors. The results of theseintermediate calculations are returned to auxiliary unit processor 870.As explained previously, auxiliary unit 70 may be attached to the mainbody of laser tracker 10 through a long cable, or it may be pulledwithin the main body of the laser tracker so that the tracker attachesdirectly (and optionally) to computer 80. Auxiliary unit 870 may beconnected to computer 80 by connection 840, which may be an Ethernetcable or wireless connection, for example. Auxiliary unit 870 andcomputer 80 may be connected to the network through connections 842,844, which may be Ethernet cables or wireless connections, for example.

Preprocessing of sensor data may be evaluated for gestures content byany of processors within the system 800, but there may also be a gesturerecognition engine 826 specifically designated to carry out gesturespreprocessing. Gestures engine 826 may include a microprocessor, DSP,FPGA, or similar device. It may contain a buffer that stores data to beevaluated for gestures content. Preprocessed data may be sent toauxiliary unit for final evaluation, or final evaluation of gesturescontent may be carried out by gestures preprocessor 826. Alternatively,raw or preprocessed data may be sent to computer 80 for analysis.

In one embodiment, the capture device 55 generates a 3D skeletal modelof the operator that allows for the interpretation of movements and orbody positions as commands to be executed by the laser tracker 10. Theskeletal model may include information, such as the position of jointson the operator and locations of specific body portions. In oneembodiment, the skeletal model identifies the location of the operator'shand, fingers and the connecting joints.

The gestures engine 826 may include a collection of gesture filters,each comprising information concerning a gesture that may be performedby the user as interpreted through the skeletal model. The data capturedby camera 59 in the form of the skeletal model and movements of theskeletal model may be compared to the gesture filters in the gestureengine 826 to identify when an operator (as represented by the skeletalmodel) has performed one or more gestures. Those gestures may beassociated with various controls of the laser tracker 10. Thus, theprocessing system 800 may use the gesture engine 826 to interpretmovements of the skeletal model and control an application based on body(e.g. hand) position or movements.

The gesture filters may be modular or interchangeable. In oneembodiment, the filter has a number of inputs, each having a type, and anumber outputs, each having a type. Inputs to the filter may compriseitems such as joint data about a user's joint position (e.g. anglesformed by the bones that meet at the joint), RGB color data, and therate of change of an aspect of the user. Outputs from the filter mayinclude parameters such as a confidence level that a particular gesturehas been made and the speed of motion of the gesture. Filters mayfurther include contextual parameters that allow for the recognition ofparticular gestures in response to previous actions.

Although embodiments that use of gestures to control the measurement ofan object describe the operation in connection with a single lasertracker, this is for exemplary purposes and the claimed invention shouldnot be so limited. In other embodiments, gestures may be used withcollections of laser trackers or with laser trackers combined with otherinstruments. One possibility is to designate one laser tracker as themaster that then sends commands to other instruments. For example, a setof four laser trackers might be used in a multilateration measurement inwhich three-dimensional coordinates are calculated using only thedistances measured by each tracker. Commands could be given to a singletracker, which would relay commands to the other trackers. Anotherpossibility is to allow multiple instruments to respond to gestures. Forexample, suppose that a laser tracker were used to relocate anarticulated arm CMM. An example of such a system is given in U.S. Pat.No. 7,804,602 to Raab, which is incorporated by reference herein. Inthis case, the laser tracker might be designated as the master in therelocation procedure. The operator would give gestural commands to thetracker, which would in turn send appropriate commands to thearticulated arm CMM.

Referring now to FIGS. 4-5, an example is shown of the operator 100making a gesture with a hand to acquire a measurement during theoperation of the laser tracker 10. The operator 100 first places theretroreflector 26 in a location where a measurement is desired, such ason object 102 for example. With the retroreflector 26 in the desiredlocation, the operator 100 raises his hand 104 and makes a predefinedhand gesture 106, such as a closed fist for example. As the operator 100moves, images of the operator 100 are acquired by the camera 59 ofcapture device 55. The projector 57 emits a light pattern onto theoperator 100 which allows the capture device 55 to determine coordinatesof points on the operator 100. From these coordinates, a skeletal modelof the operator 100 is generated. In one embodiment, a portion of theoperator, such as a hand is identified and tracked. When the operator100 makes a gesture, such as holding up a closed fist for example, inview of the camera 59, the image is acquired and compared withpredetermined gestures with the gesture engine 826. In one embodiment,this comparison is performed using filters as described herein above.Once the gesture is determined by the gesture engine 826, the processingsystem 800 initiates one or more actions in response. For example, whenthe operator 100 holds up a closed fist, the laser tracker may measureand record the three-dimensional coordinates of the retroreflector 26.The tracker may respond in a variety of ways, some not involving aretroreflector. For example, the tracker may respond by starting ameasurement sequence in which a laser beam is directed to the first of asequence of points that are to be measured sequentially. As anotherexample, the tracker may respond by carry out an initializationprocedure. As a third example, the tracker might use the cameras 52 andLEDs 54 to illuminate a collection of retroreflectors, and then measurethe three-dimensional coordinates of each in turn.

Referring now to FIG. 6, a method 110 is shown for acquiring andexecuting a command on a laser tracker 10 using a gesture. In block 112,the method 110 starts by scanning for a gesture using the capture device55. The method 110 then proceeds to block 114 where the operator makesthe gesture. The gesture is acquired by capture device 55 and parsed bygesture engine 826 in block 116. In one embodiment, the operator maymake three signals, a first gesture in block 118 to indicate a commandgesture is forthcoming, a second gesture in block 120 that indicates thecontent of the command, and a third gesture in block 122 that indicatesthe gestures are command gestures are complete. The use of a prologuegesture 118 and an epilogue gesture 122 may provide advantages insituations where there is a risk of the operator making a commandgesture unintentionally.

FIGS. 7-9 illustrate two sets of gestures 124, 126. Each of the gestures124, 126 is associated with a corresponding command 128 in the lasertracker 10. In an exemplary case, laser tracker commands includemeasuring a comp off point, measuring a comp axis point, measuring aplane, measuring a 2D line, measuring a circle, measuring a cylinder,measuring a sphere, changing an SMR, resetting an interferometer,setting a distance mode, searching for a target, toggling between singlepoint mode and scan mode, collecting a reading, moving to a homeposition, removing a reading, autoadjusting an ADM using an SMR,autoadjusting an ADM using an internal retroreflector, initializing acommand tablet, setting an SMR, and acquiring an SMR.

In one embodiment, the laser tracker 10 may include a user input device,such as a keyboard for example, that allows for a short-cut sequence ofkeystrokes 130 to execute the command. The first set of gestures 124corresponds to a motion gesture, such as where the operator holds up ahand and moves the hand in a prescribed pattern. The pattern may be intwo or three dimensions. For each of the first set of gestures 124, thestarting position is indicated with a small circle and the endingposition is indicated with an arrow. The second set of gestures 126corresponds to a static gesture wherein the operator makes apredetermined pose with a portion of their body, such as a hand gesture.It should be appreciated that the gestures may be combined together toform a compound gesture that indicates to the laser tracker 10 a seriesof commands to be performed. Further, it should be appreciated that thegestures shown in FIGS. 7-9 are exemplary and the claimed inventionshould not be so limited.

Once the capture device 55 acquires the image of the gesture and parsesthe command gesture, the method 110 may optionally acknowledge receiptof the command in block 132. In one embodiment, simultaneous withacknowledging receipt of the command, the method 110 may proceed toquery block 134 where a confidence level in the acquired gesture isdetermined. The confidence level corresponds to the likelihood that theoperator's movement or pose corresponds with the gesture. The confidencelevel may be scalable, such as from 0 to 1 for example. When theconfidence level is below a predetermined threshold, such as 95% or 0.95for example, the method loops back to block 112 and the process isrestarted. If the confidence level is above the predetermine threshold,then the method 110 proceeds to block 136 where the laser tracker 10executes the command indicated by the acquired gesture and then actuatesan action complete indicator to alert the operator the command has beencompleted.

Referring now to FIGS. 10-12, exemplary embodiments are shown ofoperating the laser tracker 10 with gestures. In FIG. 10, the method 140starts in block 142 and acquires a gesture 144, which in this examplecorresponds to the command “Set Reference Point” in block 146. Themethod 140 proceeds to block 148 where the laser tracker 10 acknowledgesthe command by flashing a colored light, such as a red light emittingdiode (LED) 149 for example, twice. A common reference position is thehome position of the laser tracker 10, which corresponds to the positionof a magnetic nest 150 permanently mounted on the body 16 of the lasertracker 10. Another reference point that is located close to the workvolume may be chosen to avoid having the operator to walk back to thelaser tracker 10 when connection the laser beam 46 to the retroreflector26 is interrupted. The operator then proceeds to place theretroreflector 26 at the reference location (e.g. nest 150) in block152. With the retroreflector 26 in the reference location, the operatormakes a gesture 154, which initiates the measurement in block 156. Thelaser tracker 10 then makes a measurement to set a reference point inblock 158. In one embodiment, when the laser tracker 10 makes ameasurement, the colored light or LED 149 is activated for apredetermined amount of time, such as five seconds for example.

In FIG. 11, a method 160 is shown for measuring a diameter of a featureon an object. The method 160 starts in block 162 and the operatorgestures 164 to measure a circle in block 166. The laser tracker 10acknowledges the command in block 168, such as by flashing the LED 149twice. The operator then proceeds to place the retroreflector 26 againstthe object in block 170 and initiates a measurement in block 172 bymaking a gesture 174. For example, if the operator is measuring theinside of a circular hole, the SMR will be placed against a surface onthe inside of the hole. The laser tracker 10 measures the coordinates ofthe point where the retroreflector is located in block 176 and indicatesthe measurement to the operator by activating the LED 149 for fiveseconds. The operator then moves the retroreflector 26 about feature inblock 178 and makes a gesture 180 to measure an inner diameter orgesture 182 to measure an outer diameter in block 184 to remove anoffset distance to account for the radius of retroreflector 26. Thelaser tracker 10 then acknowledges receipt of the command in block 186.The operator then moves the retroreflector 26 about the feature in block188. When enough points have been collected, the operator moves theretroreflector 26 away from the surface of the object.

In FIG. 12, a method 190 is shown for acquiring the retroreflector 26with the laser tracker 10. The method 190 starts in block 192 andproceeds to block 194 where the operator gestures 193 to indicate thatretroreflector should be acquired. The laser tracker 10 acknowledgesreceipt of the gesture in block 196 such as by flashing the LED 149twice. The laser tracker 10 then proceeds in block 198 to drive or steerthe laser beam 46 toward the retroreflector 26. In one embodiment, theLED 149 flashes twice and then turns green when the retroreflector isacquired. The method 190 then proceeds to query block 200 where it isdetermined whether the laser beam 46 has been directed onto theretroreflector 26. If query block 200 returns an affirmative, meaningthat the position of the retroreflector 26 has been captured, then thelaser tracker 10 indicates this by changing the LED 149 to green. If thequery block 200 returns a negative then the method 190 proceeds to block202 where a search pattern is performed, such as a spiral searchpattern. The search pattern is performed until the retroreflector 26 isacquired by the laser tracker 10. It should be appreciated that in someembodiments, the laser tracker 10 may use camera's 52 to locate thegeneral area of the retroreflector 26 to reduce the time to acquire theretroreflector 26.

Referring now to FIGS. 13 and 14 another embodiment of a laser tracker12 is shown for use with a gesture device 210. The gesture device 210may be any device or devices implemented in various form factors, whichis either worn by the operator 100 or temporarily attached to theoperator's body, such as a sleeve that is worn on the operator's forearm212 for example. The gesture device 210 provides a human computerinterface (HCI) that allows the operator to control and interact withthe laser tracker 12 via electrical signals generated by the movement ofthe user's muscles.

In one embodiment, the gesture device 210 includes a body 214 made froman elastic material that allows the gesture device 210 to fit tightly,but comfortably, about the operator's forearm 212. A plurality ofsensors 216 are coupled to the body 214 adjacent the operator's skin. Inone embodiment, the sensors 216 are electromyography (EMG) sensor nodes.The EMG sensors 216 detect an operator's muscle generating electricalsignals produced by the operator's body in response to movement, such aswhen the operator moves an arm 212 or fingers 224 for example. Thegesture device 210 measures the electrical voltage potential anddetermines the movement or position of the operator based on the musclesthat are activated and the level of the electrical signal. Thus, if theoperator makes a hand gesture, such as making a closed fist for example,the gesture device 210 can determine the gesture being made. It shouldbe appreciated that in some embodiments, the operator may calibrate thegesture device 210 to associate desired muscles with the sensors 216.

In the exemplary embodiment, the gesture device 210 includes atransmitter 218 that transmits a signal 220 to a receiver 222 in thelaser tracker 12. The transmission of the signal 220 may be performed bya suitable wireless medium, such as Bluetooth, Wi-Fi (IEEE802.11standard) for example. In another embodiment, the gesture device210 is coupled to the laser tracker 12 by a wired medium, such as aUniversal Serial Bus (USB) connection for example. The gesture device210 will also include a power source, such as batteries for example,that provide the energy to operate the gesture device 210. In oneembodiment, the determination of the gesture made by the operator 100 isperformed in the gesture device 210. It should be appreciated that wherethe gesture determination is performed in the gesture device 210, thegesture device 210 may include a controller (not shown) having aprocessor. In another embodiment, the voltage potential data istransmitted to the receiver 222 and the gesture determination isperformed in the laser tracker 12.

During operation, the operator slides the gesture device 210 onto hisforearm 212 and aligns or calibrates the gesture device 210 ifnecessary. It should be appreciated that the gesture device 210 may beplaced on either arm of the operator or that a differently shapedgesture device may be placed elsewhere on the body, such as a finger forexample. The operator then proceeds to give the desired command. In oneembodiment, the operator makes hand gestures, such as those shown inFIGS. 7-9 in the process of performing measurements with a laser tracker10. It should be appreciated that the gesture device 210 providesadvantages in that the operator's arm or hand does not need to be withina line of sight of the laser tracker 12 to transmit commands and performoperations with the laser tracker 12.

Referring now to FIGS. 15-16, another embodiment is shown of usinggestures for conveying commands to a laser tracker. In this embodiment,the laser tracker 10 includes a camera, such as camera 59 (FIG. 1A) forexample. The camera 59 has a field of view 300 that defines that volumeof space in which images may be acquired. In embodiments where thecamera 59 is attached to the payload 15, the field of view 300 will moveas the payload is rotated about the axis 18, 20. It should beappreciated that in some embodiments, the camera 59 is separate from thelaser tracker 10 and may be arranged to provide a field of view thatincludes the areas where measurements are to be performed.

As the operator moves within the field of view 300, the camera 59 maytrack the movement of the operator and distinguish the operator frombackground objects, or other moving objects (e.g. forklifts) within theacquire image. In one embodiment, a skeletal model 302 is fit to theoperator to facilitate tracking of the user. The skeletal model 302 mayinclude a first body part 304 (e.g. an arm or torso) and a second bodypart 306 (e.g. a finger). In this way, a gesture, such as pointing at anobject (e.g. a retroreflector) may be detected from the images acquiredby the camera 59. This provides advantages in allowing a single operatorto make measurements using multiple retroreflectors. In one embodiment,the laser tracker 10 determines which retroreflector 26 the operator 100desires to measure by identifying an object that the operator 100 ispointing at is a retroreflector. The laser tracker 10 may thenautomatically orient the payload 14 such that the light source 54 (FIG.1A) directs the light onto the retroreflector. Thus the laser tracker 10may determine the three-dimensional coordinates of a point on the object102 based at least in part on angular measurements by the encoders andthe time it takes for the beam of light to travel to the retroreflector26 and back to the laser tracker 10.

Referring now to FIG. 16 a method 310 is shown for tracking an operatorand conveying commands to a laser tracker. The method 310 starts inblock 312 where an image is acquired, such as by camera 59 for example.The method 310 then proceeds to block 314 where a human detection moduledetermines if the operator is within the acquired image. The humandetection module 314 may invoke a detection method, such as a shapebased detection module 316. In one embodiment, the shape based detectionmodule 316 includes a stabilization engine that aligns the current imageframe with a proceeding image frame. The foreground region may then bedetermined at least in part by subtracting the images. In oneembodiment, the subtraction is performed in the hue channel of hue,saturation and value (HSV) color space. Objects and people within theforeground may be identified by the shape based detection module 316 bycomparing edges within the image to a database of templates of shapes,such as human silhouettes for example.

If an operator is detected, the method 310 may bifurcate into twoparallel processes. In the first path 313, the operator is tracked intracking module 318. This module tracks the position of the operatorthrough successive image frames, this may be done for example bymodeling the pixel intensities over time. The method 310 also uses amotion analysis module 320 to verify that the tracked object is human.The motion analysis may be performed by performing a periodicity test onthe pixels within a bounded box of the operator being tracked. If thenumber of periodic pixels is higher than a threshold, then the trackedoperator is determined to be human and not a moving object, such as aforklift for example. The human detection, tracking and motion analysisprocesses may be similar to those described in the article “Real-TimeHuman Detection in Uncontrolled Camera Motion Environments” by Husseinet al, Institute for Advanced Computer Studies, University of Maryland(http://www.umiacs.umd.edu/˜wamageed/papers/icvs2006.pdf, accessed April25, 2014), the contents of which is incorporated herein by reference.

It should be appreciated that while the human detection module 314, thehuman tracking module 318 and the motion analysis module 320 areillustrated as being performed serially, this is for exemplary purposesand the claimed invention should not be so limited. In other embodimentsthe modules 314, 318, 320 may run in parallel or partially in parallel.

On the second path 322, the method 310 proceeds to block 324 where agesture detection module 324 determines if the detected operator hasmade a gesture. As discussed above, in one embodiment the gesturedetection is performed to detect a human pose, such as by fitting askeletal model 302 to the operator for example. In one embodiment, thelaser tracker 10 may further include a projector that emits a structuredlight towards the operator. As discussed herein above, the structuredlight is received by the camera and a three dimensional coordinates ofthe operator may be determined. In this manner, the portions of theoperators body (e.g. arm, hand, torso, legs) may be determined alongwith joints (e.g. shoulder, elbow, wrist). In another embodiment, theskeletal model 302 is applied by comparing the edge of the detectedoperator to a database of human silhouettes. The gestures that may beperformed include, but are not limited to, pointing at a retroreflector,pointing to reacquire a retroreflector after a beam break for example.The gesture detection module 324 may detect the gesture usingshape-based detection module 316.

If a gesture has been detected, the method 310 proceeds to block 326where an object detection module detects one or more objects dependingon the gesture. For example, if the gesture is the operator pointing,the object detection module determines if there are any objects along avector defined by the operators arm or finger. For example, when thegesture is a pointing finger, the object detection module may determinewhether a retroreflector is located in an area along the vector definedby the pointing finger. In one embodiment, the detection of the objectmay be determined based on the shape-based detection module 316. Itshould be appreciated that this detection may also be context sensitive.Where a laser tracker 10 has been tracking retroreflector and the beamhas been broken, the pointing of the finger may indicate that the lasertracker 10 is to require the retroreflector that the operator ispointing to. In other embodiments, the gesture may indicate otherobjects, such as the object to be measured. Once the object has beendetected, the method 310 proceeds to block 328 where the laser trackercommand is executed, such as measuring the distance to theretroreflector, reacquire and track the retroreflector, or perform aseries of predetermined measurements on an object for example.

Although the gesture devices described hereinabove are directed inexemplary embodiments to use with a laser tracker, it should beunderstood that such a gesture tracking device may be used with anothertype of three-dimensional measurement device such as a laser tracker, atriangulation type scanner, a time-of-flight type scanner, aphotogrammetry system, or a Cartesian coordinate measurement machine(CMM), for example.

Technical effects and benefits of embodiments of the present inventioninclude the visual conveying of commands from an operator to a metrologyinstrument, such as a laser tracker for example. The conveying ofcommands allows for an operator to initiate actions and operations onthe metrology instrument at a distance with no physical orradio/wireless connection.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

While the invention has been described in detail in connection with onlya limited number of embodiments, it should be readily understood thatthe invention is not limited to such disclosed embodiments. Rather, theinvention can be modified to incorporate any number of variations,alterations, substitutions or equivalent arrangements not heretoforedescribed, but which are commensurate with the spirit and scope of theinvention. Additionally, while various embodiments of the invention havebeen described, it is to be understood that aspects of the invention mayinclude only some of the described embodiments. Accordingly, theinvention is not to be seen as limited by the foregoing description, butis only limited by the scope of the appended claims.

What is claimed is:
 1. A method for a user to control operation of alaser tracker with steps comprising: providing the laser tracker havinga structure rotatable about a first axis and a second axis, a firstlight source that launches a first light beam from the structure, adistance meter, a first angular encoder that measures a first angle ofrotation about the first axis, a second angular encoder that measures asecond angle of rotation about the second axis, and a processor;providing a second light source and a camera, the second light sourceconfigured to project a second light beam, the camera configured toacquire an image of the second light beam reflected off of a user;providing a rule of correspondence between each of a plurality ofcommands and each of a plurality of gestures, each gesture from amongthe plurality of gestures corresponding to a body position of the user;projecting the second light beam from the second light source andacquiring with the camera a first image of the user; generating askeletal model of the user from the first image; performing by the usera first gesture from among the plurality of gestures, the first gesturecorresponding to a first body position; identifying from the skeletalmodel the performing by the user of the first gesture; determining thefirst command based at least in part on the identifying of the firstgesture according to the rule of correspondence; and executing the firstcommand with the laser tracker.
 2. The method of claim 1 wherein thesecond light beam includes a structured light pattern, and the imageincludes at least a portion of the structured light pattern.
 3. Themethod of claim 1 wherein the generating the skeletal model includesdetermining three dimensional coordinates of the user's body.
 4. Themethod of claim 1 further comprising: providing a plurality of gesturefilters, each gesture filter including information of a gestureperformed by the user as interpreted through the skeletal model, whereinthe identifying of the first gesture is based at least in part on one ofthe plurality of gesture filters.
 5. The method of claim 4 wherein eachof the plurality of gesture filters include a plurality of inputs and aplurality of outputs.
 6. The method of claim 5 wherein the plurality ofinputs includes at least one of joint data, RGB color data and a rate ofchange of an aspect of the user.
 7. The method of claim 5 wherein theplurality of outputs includes at least one of confidence level and aspeed of motion of the gesture.
 8. The method of claim 1 furthercomprising: performing by the user a second gesture from among theplurality of gestures, the second gesture corresponding to a second bodyposition; identifying from the skeletal model the performing by the userof the second gesture; determining the second command based at least inpart on the first gesture and the identifying of the second gestureaccording to the rule of correspondence; and executing the secondcommand with the laser tracker.
 9. The method of claim 1 wherein thefirst command is to acquire a retroreflector.
 10. The method of claim 9further comprising: acquiring a second image with the camera;identifying an area containing the retroreflector; and rotating thelaser tracker about at least one of the first axis and second axis toacquire the retroreflector based at least in part on the identifying ofthe area.
 11. The method of claim 9 further comprising performing withthe laser tracker a spiral search pattern to acquire the retroreflector.12. The method of claim 1 wherein the first body position includes apointing of a finger by the user at an object.
 13. The method of claim12 further comprising: identifying based at least in part on theskeletal model the object in the first image the user is pointing at;and rotating the laser tracker about the first axis and second axis toproject the first light beam toward the object.
 14. The method of claim13 wherein the object is a first retroreflector.
 15. The method of claim14 further comprising a plurality of retroreflectors, wherein theplurality of retroreflectors includes the first retroreflector.
 16. Themethod of claim 13 further comprising: identifying based at least inpart on the skeletal model a point on the object the user is pointingat; and determining three-dimensional coordinates of the point.
 17. Themethod of claim 1 further comprising: determining that the user is inthe first image; acquiring with the camera a second image; aligning thesecond image with the first image; and identifying foreground shapes.18. The method of claim 17 wherein at least one of the foreground shapesis the user.
 19. The method of claim 18 further comprising subtractingthe first image from the second image to identify foreground shapes. 20.The method of claim 18 further comprising tracking a movement of theuser from the first image to the second image.